Information processing device and information processing method

ABSTRACT

In an information processing device, an acquisition unit acquires information on a position of an occupant from a vehicle that is in an accident. A reception unit receives a manipulation input from an operator. A determination unit determines directivity of a microphone in a vehicle cabin of the vehicle based on the manipulation input received by the reception unit.

INCORPORATION BY REFERENCE

This application is a continuation of U.S. patent application Ser. No.16/538,892, filed February Aug. 13, 2019, which claims the benefit ofJapanese Patent Application No. 2018-200397 filed on Oct. 24, 2018including the specification, drawings and abstract is incorporatedherein by reference in its entirety.

BACKGROUND 1. Technical Field

The present disclosure relates to an information processing device andan information processing method for acquiring information on a vehiclethat is in an accident.

2. Description of Related Art

A system in which, when a collision accident has occurred in a vehicle,an emergency notification is automatically executed from an emergencynotification device mounted in the vehicle to a management center, andan ambulance or the like is caused to be directed to an accident site byarrangement of the management center has been known.

Japanese Unexamined Patent Application Publication No. 2016-30481 (JP2016-30481 A) discloses a vehicular emergency notification devicecapable of estimating a degree of injury of an occupant on themanagement center side when a vehicle accident occurs. In this device,microphones are provided near seats of the vehicle one by one, a voiceof the occupant is acquired by the microphone, and voice information istransmitted to the management center. An operator of the managementcenter ascertains, for example, a situation of injury by talking withthe occupant.

SUMMARY

In a technology of JP 2016-30481 A, for example, clear acquisition of avoice of a specific occupant is likely to be difficult in a situation inwhich noise outside an accident vehicle is great.

The present disclosure has been made in view of such circumstances, andthe present disclosure provides an information processing device and aninformation processing method capable of facilitating clear acquisitionof a voice of an occupant at a designated position in a vehicle cabinwhen an accident has occurred in a vehicle.

A first aspect of the present disclosure relates to an informationprocessing device. The information processing device includes anacquisition unit configured to acquire information on a position of anoccupant from a vehicle that is in an accident; a reception unitconfigured to receive a manipulation input from an operator; and adetermination unit configured to determine directivity of a microphonein a vehicle cabin of the vehicle based on the manipulation input.

According to the first aspect, since the directivity of the microphonein the vehicle cabin of the vehicle that is in the accident isdetermined based on the manipulation input of the operator, it ispossible to facilitate clear acquisition of the voice of the occupant atthe designated position.

In the information processing device according to the first aspect, theacquisition unit may acquire information on the occupant from thevehicle, and the information processing device may include a displayunit that displays an image representing a seating position of theoccupant and a state of the occupant based on the information on theoccupant.

In the information processing device according to the first aspect, thereception unit may receive a manipulation input for designating theseating position of the occupant in the image displayed on the displayunit, and the determination unit may determine the directivity of themicrophone to be a direction of the designated seating position of theoccupant.

In the information processing device according to the first aspect, theacquisition unit may acquire information on an open and closed state ofa window from the vehicle, and the display unit may display an imagerepresenting the open and closed state of the window.

In the information processing device according to the first aspect, thedetermination unit may determine the directivity of the microphone to bea direction of a designated window based on a manipulation input fordesignating the window in the image displayed on the display unit.

A second aspect of the present disclosure also relates to an informationprocessing device. The information processing device includes anacquisition unit configured to acquire information on an occupant from avehicle that is in an accident; and a display unit configured to displayan image representing a seating position of the occupant and a state ofthe occupant based on the information on the occupant.

According to the second aspect, it is possible to rapidly ascertain theseating position and the state of each occupant of the vehicle that isin the accident based on the image.

A third aspect of the present disclosure relates to an informationprocessing method. The information processing method includes anacquisition step of acquiring information on a position of an occupantfrom a vehicle that is in an accident; a reception step of receiving amanipulation input from an operator; and a determination step ofdetermining directivity of a microphone in a vehicle cabin of thevehicle based on the manipulation input received in the reception step.

According to the aspects of the present disclosure, it is possible tofacilitate clear acquisition of a voice of an occupant at a designatedposition in a vehicle cabin when an accident has occurred in a vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, advantages, and technical and industrial significance ofexemplary embodiments of the disclosure will be described below withreference to the accompanying drawings, in which like numerals denotelike elements, and wherein:

FIG. 1 is a block diagram illustrating a configuration of an emergencycall system according to an embodiment;

FIG. 2 is a diagram schematically illustrating a vehicle cabin of avehicle in FIG. 1; and

FIG. 3 is a diagram illustrating an example of an image displayed on adisplay unit of an information processing device in FIG. 1.

DETAILED DESCRIPTION OF EMBODIMENTS

FIG. 1 is a block diagram illustrating a configuration of an emergencynotification system 1 according to an embodiment. The emergencynotification system 1 includes an emergency notification device 10 andan information processing device 40.

The emergency notification device 10 is mounted in a vehicle 90 that isa car. The emergency notification device 10 has a wireless communicationfunction, and is connected to a network 30 via a wireless base stationor a wireless access point. An information processing device 40 isconnected to the network 30, and the information processing device 40communicates with the emergency notification device 10 via the network30. The information processing device 40 is installed, for example, atan emergency notification center and used by an operator. A standard ofthe wireless communication is not particularly limited, and includes,for example, 3G (a third generation mobile communication system), 4G (afourth generation mobile communication system), or 5G (a fifthgeneration mobile communication system).

The vehicle 90 includes an emergency notification device 10, an occupantdetection sensor 12, a seat belt sensor 14, a window opening and closingsensor 16, a door ECU 18, a microphone 20, and a speaker 22. Theemergency notification device 10 includes a detection unit 50, anacquisition unit 52, a derivation unit 54, a communication unit 56, aholding unit 58, a directivity controller 60, and a call unit 62.

The occupant detection sensor 12 is provided at each seat of the vehicle90, detects a load on a seat to detect a seating state of the occupanton each seat, and outputs a detection result to the acquisition unit 52.

The seat belt sensor 14 is provided in each seat of the vehicle 90,detects a wearing state of the seat belt of the occupant on each seat,and outputs a detection result to the acquisition unit 52.

The window opening and closing sensor 16 detects an open and closedstate of each window of the vehicle 90, and outputs a detection resultto the door ECU 18 and the acquisition unit 52.

When an acceleration sensor (not illustrated) detects an accelerationequal to or greater than a predetermined threshold value due to, forexample, a collision of the vehicle 90, an airbag electronic controlunit (ECU) (not illustrated) outputs a deployment signal for deployingan airbag to the airbag and the detection unit 50. When the detectionunit 50 receives the deployment signal, the detection unit 50 detectsthat an accident has occurred in the vehicle 90.

When the occurrence of the accident is detected by the detection unit50, the door ECU 18 diagnoses whether or not there is a possibility thatthe window is broken with respect to the respective windows of which aclosed state has been detected by the window opening and closing sensor16. The door ECU 18 sets a jam protection function to be effective,closes the window, and executes the diagnosis. The jam protection is afunction of executing a window opening operation when a load is appliedto a motor for opening and closing a window during a window closingoperation to prevent pinching of a hand or the like. Normally, the jamprotection function is disabled when the window is substantiallycompletely closed, but is enabled for diagnosis. When the window is notbroken, it is assumed that a load is applied to the motor through anoperation of closing a window that has already been closed and the jamprotection works. When the window is broken, it is assumed that no loadis applied to the motor through an operation of closing the window andthe jam protection does not work. The door ECU 18 detects that thewindow is in an open state when the jam protection does not work in thewindow in which the window opening and closing sensor 16 indicates aclosed state, and outputs a detection result to the acquisition unit 52.

The acquisition unit 52 regularly acquires the detection result of theoccupant detection sensor 12 and the detection result of the seat beltsensor 14 regardless of the presence or absence of an accidentoccurrence detection. When occurrence of an accident is detected by thedetection unit 50, the acquisition unit 52 outputs the detection resultof the occupant detection sensor 12 and the detection result of the seatbelt sensor 14 acquired immediately before the occurrence of theaccident to the derivation unit 54.

When the occurrence of the accident is detected by the detection unit50, the acquisition unit 52 acquires the detection result of the openand closed state of the window output from at least one of the door ECU18 or the window opening and closing sensor 16, and outputs the acquireddetection result to the derivation unit 54. As described above, in thewindow in which the detection result of the window opening and closingsensor 16 indicates a closed state, the window is considered to be openwhen the detection result of the door ECU 18 indicates an open state.

When the detection unit 50 detects occurrence of an accident, thederivation unit 54 derives the emergency notification information andoutputs the derived emergency notification information to thecommunication unit 56. The emergency notification information is alsoreferred to as minimum set of data (MSD). The emergency notificationinformation includes information for specifying the vehicle 90, theposition of the vehicle 90, the detection result of the occupantdetection sensor 12, the detection result of the seat belt sensor 14,and the detection result of the open and closed state of the window,which are output from the acquisition unit 52. The information forspecifying the vehicle 90 includes a vehicle identification number(VIN), information on a license plate, a vehicle type, a color of thevehicle 90, and the like, and is stored in a storage unit (notillustrated) in advance. The position of the vehicle 90 is acquired by aGPS receiver (not illustrated).

The communication unit 56 transmits the emergency notificationinformation output from the derivation unit 54 to the informationprocessing device 40.

FIG. 2 schematically illustrates a vehicle cabin of the vehicle 90 inFIG. 1. One microphone 20 is provided, for example, near a center of thevehicle cabin. The microphone 20 is unidirectional, and a sensitivity ina specific direction is higher than a sensitivity in other directions.In a normal time other than the time of occurrence of an accident, thedirectivity of the microphone 20 is set to a direction d1 of a driver'sseat so that it is easy to acquire driver's speech. In the normal time,information on voice acquired by the microphone is supplied to anavigation device (not illustrated) or the like, and the navigationdevice or the like can be manipulated through voice recognition. Thedirection of the directivity of the microphone can be changed under thecontrol of the directivity controller 60. The directivity of themicrophone 20 can be changed to, for example, a direction d2 of apassenger seat, a direction d3 of a right rear seat, a direction d4 of aleft rear seat, a direction d5 of the window on the driver's seat side,and the like.

Return to FIG. 1. The holding unit 58 holds control information for thedirectivity of the microphone 20 regarding each seat and each window inadvance. The directivity control information includes, for example, anangle for setting the directivity in a direction of each seat and eachwindow.

The directivity controller 60 controls the directivity of the microphone20 based on directivity information of the microphone 20 transmittedfrom the information processing device 40 to be described below at thetime of occurrence of an accident and control information for thedirectivity held in the holding unit 58. For example, when thedesignated directivity is a direction of the right rear seat, thedirectivity controller 60 controls the directivity based on the controlinformation for the directivity on the right rear seat.

Voice information acquired by the microphone 20 is also supplied to thecall unit 62. When the occurrence of the accident is detected, the callunit 62 is connected to the call unit 78 of the information processingdevice 40 via the communication unit 56, and executes a call between theoccupant of the vehicle 90 and the operator of the informationprocessing device 40 using the microphone 20 and the speaker 22. Thespeaker 22 is provided in the vehicle cabin and outputs a voice based ona voice signal supplied from the call unit 62.

The information processing device 40 includes a communication unit 70,an acquisition unit 72, a display controller 74, a display unit 76, acall unit 78, a reception unit 80, and a determination unit 82. Thecommunication unit 70 receives the emergency notification informationfrom the vehicle that is in the accident, and outputs the receivedemergency notification information to the acquisition unit 72.

The acquisition unit 72 acquires the emergency notification informationoutput from the communication unit 70, and acquires information on theoccupant of the accident vehicle based on the emergency notificationinformation.

The acquisition unit 72 acquires information on the position of theoccupant based on the detection result of the occupant detection sensor12 in the emergency notification information.

The acquisition unit 72 acquires information on the state of theoccupant based on the detection result of the occupant detection sensor12 and the detection result of the seat belt sensor 14 in the emergencynotification information. The acquisition unit 72 acquires informationon the state of the occupant indicating that a degree of serious injuryof the occupant is low when the occupant is seated on a certain seat andthe seat belt is worn. The acquisition unit 72 acquires information onthe state of the occupant indicating that the degree of serious injuryof the occupant is high when the occupant is seated on the certain seatand the seat belt is not worn.

The acquisition unit 72 acquires information on the open and closedstate of the window based on the detection result of the open and closedstate of the window in the emergency notification information. Theacquisition unit 72 outputs the acquired information on the occupant andthe acquired information on the open and closed state of the window tothe display controller 74.

The display controller 74 controls the display of the image in thedisplay unit 76 based on the information on the occupant and theinformation on the open and closed state of the window, which are outputfrom the acquisition unit 72.

Under the control of the display controller 74, the display unit 76displays an image illustrating the seating position of the occupant inthe accident vehicle, the state of the occupant, and the open and closedstate of the window.

FIG. 3 illustrates an example of an image 100 displayed on the displayunit 76 of the information processing device 40. The image 100 includesa FIG. 102 indicating an occupant of which a seating position is adriver's seat, a FIG. 104 indicating an occupant of which a seatingposition is a passenger seat, and a FIG. 106 indicating an occupant ofwhich a seating position is a right rear seat. Since no figure isdisplayed in a position corresponding to the left rear seat, thisindicates that no occupant is seated at the left rear seat. Instead ofthe figures, characters indicating occupants may be displayed.

Each of the FIGS. 102, 104, and 106 also indicates a state of theoccupant using color or a pattern. The FIG. 102 indicates that a stateof the occupant at the driver's seat is “a high degree of seriousinjury”. The FIG. 104 indicates that the state of the passenger at thepassenger seat is “a high degree of serious injury”. The FIG. 106indicates that the state of the occupant at the right rear seat is “alow degree of serious injury”.

The image 100 also includes a FIG. 110 indicating that a window on thepassenger seat side is in an open state. Since no figures are displayedin positions corresponding to other windows, the image 100 indicatesthat the other windows are in a closed state.

Return to FIG. 1. When the acquisition unit 72 acquires the emergencynotification information, the call unit 78 executes a call between theoccupant of the vehicle 90 and the operator via the call unit 62 of theemergency notification device 10 that has transmitted the emergencynotification information.

The reception unit 80 receives a manipulation input of an operator fordesignating the seating position of the occupant or the position of thewindow in the open state in the image displayed on the display unit 76.The reception unit 80 may include a touch sensor that receives a touchmanipulation input on a screen of the display unit 76 by the operator.

The determination unit 82 determines the directivity of the microphone20 in the vehicle cabin of the vehicle 90 based on the manipulationinput. The determination unit 82 determines the directivity of themicrophone 20 in the direction of the designated seating position of theoccupant based on the manipulation input for designating the seatingposition of the occupant in the image displayed on the display unit 76.The determination unit 82 determines the directivity of the microphone20 in the direction of the designated window based on the manipulationinput for designating the window in the image displayed on the displayunit 76.

The determination unit 82 outputs information on the determineddirectivity of the microphone 20 to the communication unit 70, and thecommunication unit 70 transmits directivity information of themicrophone 20 to the emergency notification device 10. The emergencynotification device 10 controls the directivity of the microphone 20according to the directivity information as described above.

When a call is started, the operator calls the occupant of the vehicle90 such as “Is it OK?”. A voice of the operator is listened to at eachseat. When there is a response indicating that there is no problem fromthe driver to which the microphone 20 is directed, the operator does notrequest an ambulance.

When there is no response to the call from the driver, the operatorconfirms the image 100 on the display unit 76, and designates theseating position of the occupant with a low degree of serious injurythrough, for example, a touch manipulation on the screen. The operatorcan rapidly ascertain the degree of serious injury and the seatingposition of each occupant by confirming the image. Accordingly, itbecomes easy to start talking with an occupant with a low degree ofserious injury who is more highly likely to be able to talk than anoccupant with a high degree of serious injury. In the example of FIG. 3,it is assumed that the operator designates the right rear seat.

When the operator can talk with a designated occupant, the operatorlistens to a degree of injury of each occupant and requests anambulance. Since the microphone 20 has high sensitivity in a directionof the designated occupant and low sensitivity in the other directions,it is easy to clearly acquire the voice of the designated occupant evenin a situation in which noise outside the vehicle is large.

The operator also requests the ambulance even when there is no responsefrom all occupants. When the operator requests the ambulance, theoperator sends a notification to a communication control room of a firedepartment headquarter, informs of the position of the vehicle 90 andthe information for specifying the vehicle 90 based on the receivedemergency notification information, and informs of a degree of injurywhen the operator can listen to the degree of injury from the occupant.The information processing device 40 may transmit the emergencynotification information to a terminal device such as the communicationcommand room of the fire department.

The operator may designate the position of the window in an open statein the image when there is no response from all the occupants. Theoperator can rapidly ascertain the window in an open state by confirmingthe image. Since the directivity of the microphone 20 is controlled in adirection of the designated open window and the operator can listen tothe sound outside the vehicle collected by the microphone 20, a range ofmeasures, for example, is widened as will be shown below.

For example, even when it is difficult to specify a road on which anaccident has occurred from a position of the vehicle 90 in the emergencynotification information due to an expressway or a highway and generalroads being vertically in parallel, the road on which the accident hasoccurred is likely to be rapidly specified. When travel sound of anothervehicle is heard from the outside of the vehicle and a voice of apasserby is not heard, an accident is likely to have occurred on anexpressway or a highway. When the voice of the passerby from the outsideof the vehicle is heard, an accident is likely to have occurred on ageneral road. The operator can talk to a passerby outside the vehicle toconfirm when a road is a general road. When the operator talks to apasserby outside the vehicle, the information processing device 40 maytransmit an instruction to increase a volume of the speaker 22 to theemergency notification device 10 according to the manipulation input ofthe operator.

Further, the operator can talk with a passerby outside the vehicle torequest the passerby to perform an emergency measure of the occupant orlisten to a more detailed accident situation or the like from thepasserby. Since the microphone 20 has high sensitivity in the directionof the designated window in an open state and low sensitivity in theother directions, it is easy to clearly acquire a voice of the passerby.

Although configurations of the detection unit 50, acquisition unit 52,the derivation unit 54, the directivity controller 60, the call unit 62,the acquisition unit 72, the display controller 74, the call unit 78,the reception unit 80, and the determination unit 82 can be realized byhardware such as a CPU, a memory, or another LSI of any computer and canbe realized by software such as a program loaded into the memory,functional blocks realized by cooperation of these are depicted.Therefore, it is understood by those skilled in the art that thesefunctional blocks can be realized in various forms by solely hardware,solely software, or a combination thereof.

According to the embodiment, since the directivity of the microphone 20in the vehicle cabin of the vehicle 90 causing the accident isdetermined based on a manipulation input of the operator, it can be easyto clearly acquire the voice of the passenger in the designatedposition. Thus, it is possible to allow the operator to rapidly take anappropriate action with respect to the accident.

The present disclosure has been described above based on the embodiment.The embodiment is merely an example, and it is understood by thoseskilled in the art that various modification examples can be made withrespect to a combination of respective components or respectiveprocesses, and such modification examples are also within the scope ofthe present disclosure.

For example, in the case of the vehicle 90 in which the occupantdetection sensor 12 and the seat belt sensor 14 are not provided at therear seat, the presence or absence of opening and closing of the door onthe rear seat may be detected by a door opening and closing sensor (notillustrated), and when the door is opened and subsequently closed beforetravel of the vehicle 90 starts, the acquisition unit 52 of theemergency notification device 10 may acquire that the occupant is seatedon the seat corresponding to the door. Further, when the occupantdetection sensor 12 and the seat belt sensor 14 are not provided at therear seat, the directivity of the microphone 20 may be regularlydirected to a direction of the rear seat during travel of the vehicle90, and the acquisition unit 52 may estimate a seating position of theoccupant at the rear seat based on a frequency of a sound collected bythe microphone 20 and the direction of the directivity when the sound iscollected. In these modification examples, in the vehicle 90 in whichthe occupant detection sensor 12 and the seat belt sensor 14 are notprovided at the rear seat, it is possible to cause the operator toascertain the presence or absence of the occupant at the rear seat orthe seating position.

Although one microphone 20 is provided in the vehicle 90 in theembodiment, a first microphone of which the directivity can becontrolled with respect to the occupant at the driver's seat, theoccupant at the passenger seat, and the window, and a second microphoneof which the directivity can be controlled with respect to the occupantat the rear seat and the window can be provided. In this modificationexample, a degree of freedom of the configuration of the vehicle 90 canbe improved.

The emergency notification device 10 may hold driver information inputin advance. The driver information includes, for example, informationlikely to be effective for lifesaving treatment, such as an age group, ablood type, and a sex. When the occurrence of an accident is detected,the derivation unit 54 also includes the driver information in theemergency notification information. In this modification example, forexample, even when the driver is not conscious, the operator can informemergency personnel before a movement of the driver information, and itbecomes easy for the emergency personnel to prepare for a movement.

The derivation unit 54 of the emergency notification device 10 may alsoinclude, in the emergency notification information, information on anacceleration at the time of occurrence of an accident detected by anacceleration sensor (not illustrated). The acquisition unit 72 of theinformation processing device 40 may increase the degree of seriousinjury when an absolute value of the acceleration is greater for theoccupant on the seat in which the seat belt is not worn. That is, thedegree of serious injury may have steps more than two steps includinghigh and low. In the modification example, it is possible to increasethe amount of information to be provided to the operator.

When rollover of the vehicle 90 is detected by a gyro sensor (notillustrated) or the like, the derivation unit 54 of the emergencynotification device 10 may also include rollover information in theemergency notification information. In the modification example, it ispossible to increase the amount of information to be provided to theoperator.

1. An information processing device, comprising: an acquisition unitconfigured to acquire information on an open and closed state of awindow of a vehicle that is in an accident; a reception unit configuredto receive a manipulation input from an operator external to thevehicle; a display unit configured to display an image representing theopen and close state of the window based on the information on the openand closed state of the window; and a determination unit configured todetermine directivity of a microphone in a vehicle cabin of the vehicleto be a direction of a designated window based on the manipulation inputfor designating the window in the image displayed on the display unit.2. The information processing device according to claim 1, furthercomprising: a speaker in the vehicle cabin configured to output a voiceof the operator to the vehicle cabin and outside of the vehicle.
 3. Aninformation processing method, comprising: an acquisition step ofacquiring information on an open and closed state of a window of avehicle that is in an accident; a reception step of receiving amanipulation input from an operator external to the vehicle; displayingon a display unit an image representing the open and close state of thewindow based on the information on the open and closed state of thewindow; and a determination step of determining directivity of amicrophone in a vehicle cabin of the vehicle to be a direction of adesignated window based on the manipulation input for designating thewindow in the image displayed on the display unit.
 4. The informationprocessing method according to claim 3, further comprising: outputting avoice of the operator to the vehicle cabin and outside of the vehiclevia a speaker in the vehicle cabin.